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Abstract 

We provide experimental evaluation of a number of known and new algorithms for approx- 
imate computation of Monroe's and Chamberlin-Courant's rules. Our experiments, conducted 
both on real-life preference-aggregation data and on synthetic data, show that even very simple 
and fast algorithms can in many cases find near-perfect solutions. Our results confirm and com- 
plement very recent theoretical analysis of Skowron et al., who have shown good lower bounds 
on the quality of (some of) the algorithms that we study. 



1 Introduction 

Many countries are governed using indirect democracy, where the people do not make decisions 
directly, but rather select representatives (e.g., a parliament, a senate, a congress) who rule in their 
interest. Unfortunately, relatively little effort was so far invested in the algorithmic study of proce- 
dures for electing committees of representatives (few exceptions include papers 11215 1911410 . Here, 
we consider two particularly appealing rules for electing a set of representatives, namely those of 
Monroe and of Chamberlin and Courant, and we argue that while these rules in the worst case 
scenario may be difficult to compute H5I141 . in practice, very simple and efficient algorithms find 
almost-perfect approximate results. 

There are several ways in which countries can choose their parliaments (or, more generally, in 
which societies can choose committees of representatives). Often, voters are divided into districts 
and in each district we hold a local election. For the case of single-representative districts, in 
each district we have a single-winner election held according to one of the standard, well-known, 
rules such as the Plurality rule or Borda's rule. In particular, if the Plurality rule is used then 
this system is known as First-Past-the-Post (FPP): In each district the candidate supported by the 
largest number of voters is elected. However, FPP has a number of drawbacks. For example, it 
is possible that in a country with two major parties, A and B, even if 49% of the citizens support 
party B, only members of party A enter the parliament (this happens if in each district party A has a 
slight advantage over party B). Indeed, under FPP the election organizers are particularly tempted 
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to tamper with the partition of voters into districts. To circumvent this problem one might use 
multi-representative districts, where elections are held using some multi-winner voting rule (e.g., 
using Single Transferable Vote (STV)S or using a voting rule that assigns scores to the candidates 
and picks a group of those with highest scores). However, this approach only partially solves the 
problem. Further, compared to single-representative districts, multi-representative districts loosen 
the connection between the candidates and the voters that have elected them. 

Fortunately, there is a very attractive way to avoid the problems mentioned above: Instead 
of using fixed districts, we may partition the voters dynamically, based on the votes that they cast. 
Indeed, this is exactly the idea behind the fully proportional representation rules of Monroe [10] and 
of Chamberlin and Courant [31. If we seek a parliament of K representatives, then Monroe's rule 
says that we should pick a set of K candidates for whom there is an assignment of these candidates 
to the voters such that: (a) each candidate is assigned to roughly the same number of voters, and 
(b) the total satisfaction of the voters (measured in one of the ways introduced later) is maximal. 
Chamberlin-Courant's rule is similar except that it allows each selected candidate to be matched to 
a different number of voters. (Thus if one were to elect a parliament using Chamberlin-Courant's 
rule then one should use weighted voting within the parliament, weighted by the number of voters 
matched to each representative.) 

In the above description we focus on political elections, but we mention that both Monroe's rule 
and Chamberlin-Courant's rule have many different applications as well. For example, Skowron 
et al. lfl5l have presented several (multi-agent) resource allocation settings that can be modeled 
using these rules and Lu and Boutilier [5 | have proposed to use Chamberlin-Courant's rule for 
constructing recommendations for groups of agents. 

Unfortunately, computing Monroe's and Chamberlin-Courant's rules is both NP-hard 1151141 
and difficult in the parametrized sense [2 1. Thus using them in practice might simply be impossible. 
The goal of this paper is to show that not all is lost. We provide experimental evaluation of a number 
of known and new algorithms for approximate computation of Monroe's and Chamberlin-Courant's 
rules. Our experiments, conducted both on real-life preference-aggregation data and on synthetic 
data, show that even very simple and fast algorithms can in many cases find near-perfect solutions. 
Our results confirm and complement very recent theoretical analysis of Skowron et al. [15], who 
have shown good lower bounds on the quality of (some of) our algorithms. While for single-winner 
rules using approximate algorithms may be debatable, for the case of electing a large body of repre- 
sentatives, e.g., a parliament, using approximation algorithms seems far easier to justify. Indeed, a 
good approximate solution for Monroe's or Chamberlin-Courant's rule represents the society almost 
as well as a perfect solution would. 

The paper is organized as follows. In Section [2] we formally define Monroe's and Chamberlin- 
Courant's rules. In Section|3]we give an overview of the algorithms that we evaluate and in SectionH] 
we describe the data sets that we use in our experiments. Section [5] contains our main results. We 
conclude in Section [6] 

'STV for more than one winner is sometimes referred to as "Alternative Vote" (AV). In a recent referendum Great 
Britain rejected AV as a method for choosing its members of parliament. 
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2 Preliminaries 



In this section we briefly review basic notions regarding social choice theory and we define Mon- 
roe's [10] and Chamberlin and Courant's [3 | proportional representation systems. We assume the 
reader is familiar with standard notions regarding algorithms. For each positive integer n, by [n] we 
mean the set {1, . . . , n}. 

Elections. We consider elections over a given set A = {a±, . . . , a m } of alternatives. We have a set 
N = [n] of agents (the voters), where each voter i, 1 < i < n, has a preference order y i over A. A 
preference order of an agent i is a linear order over the set A; the maximal element is this agent's 
most preferred alternative, the minimal element is this agent's least preferred alternative, and the 
alternatives in the middle represent the agent's spectrum of preference. We refer to the collection 
V = (>-i,...,>- n )as the preference profile for a given election. 

Let us fix an agent i, 1 < i < n, and an alternative a £ A. By posj(a) we mean the position a 
has in i's preference order. If a is i's most preferred candidate then posj(a) = 1, and if a is i's least 
preferred candidate then posj(a) = \\A\\ = m. 

Positional Scoring Functions. Let m be the number of candidates in eleciton. A positional scor- 
ing function (PSF) is any function a: [m] — > N that satisfies the following two conditions: (a) 
a(m) = 0, and (b) for each i, j, 1 < % < j < m, a(i) > a(j). In Monroe's and in Chamberlin- 
Courant's proportional representation rules we will match agents to the alternatives that represent 
them. Intuitively, a(i) is the amount of satsifaction that an agent derives from being represented by 
an alternative that this agent ranks on the i'th position. In this paper we focus on Borda count PSF, 
which for m alternatives is defined as ag orda (i) = m — i. However, occasionally we will consider 
other PSFs as well. 

In our algorithms we assume that the PSF a to be used is given explcitly, as a vector 
(qi, . . . , a m ) of integers such that for each i, 1 < i < m, a(i) = oti. We will implcitly assume that 
the number of alternatives matches the domain of the given PSF. 

Proportional Representation. Let A = {ai, . . . , a m } be the set of alternatives and N = [n] be the 

set of agents (with preference orders over ^4). A representation function is any function : N — > A. 
For an m-candidate PSF a and a representation function <£, <£'s satisfaction is defined as: 

n 

a($) =^a( P o Si ($(i))). 

i=i 

Let K be a positive integer. A K-CC-representation function is any representation function 
$ such that ||$ _1 (A r )|| < K (that is, any representation function that matches voters to at 
most K alternatives). A iv'-Monroe-representation function <3? is any ET- CC-representation func- 
tion that additionally satisfies the following requirement: For each a £ A it holds that either 
Lj^J < 11^* ( a )ll ^ Till or ll^ >_1 ( a )ll = (^at is, each alternative represents either roughly 
agents or none of them). 

We will also consider partial representation functions. A partial CC-representation function is 
defined in the same way as a regular one, except that it may assign a null alternative, _L, to some 
of the agents. By convention, we take that for each agent i we have poSj(_L) = m. A partial 
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Monroe-representation function is defined analogously: It may assign the null alternative to some 
voters (there are no constraints on the number of agents to whom the null alternative is assigned) 
but it must be possible to extend it to a regular Monroe-representation function by replacing the 
occurrences of the null alternative with the real ones. 

We now define Monroe's and Chamberlin-Courant's (CC) rules. 

Definition 1. Let R be a member of {Monroe, CC}. Let A = {a±, . . . , a m } be a set of alter- 
natives, N = [n] be a set of agents, and a be an m-candidate PSF. Let K be the size of the set 
of representatives that we seek (K < m). We say that a K -element set W, W C A, is a set of 
a-R winners if there exists a K-R-representation function $ : N — > W such that for every other 
K-R-representation function \I/ it holds that Qi($) > a(^). 

We point out that for both Monroe's and Chamberlin-Courant's rule there may be several dif- 
ferent winner sets and that some form of tie-breaking should be applied in these settings. Here we 
disregard tie-breaking and simply are interested in some winner set (and, not being able to compute 
that, in any set with as high a satsifaction as possible). 

It is well-known that for many natural families of PSFs, both for Monroe's rule and for 
Chamberlin-Courant's rule, it is NP-complete to decide if there exists a winner set that achieves 
a given satisfaction 112151141 . However, for each R in {Monroe, CC}, for each PSF a (with the 
domain matching the number of alternatives in the election), and for each set S of up to K alterna- 
tives we can compute in polynomial time a (possibly partial) K- ^-representation function that 
maximizes the agent satisfaction under the condition that agents are matched to the alternatives in 
S only. Indeed, it is easy to see that for a-CC this function is: 

*cc(») = argmin ae5 pos i (a) 

and that it is never a partial representation function. For the case of a-Monroe, computing ^Monroe 
is more involved and requires solving a certain min-cost/max-flow problem (see the work of Betzler 
et al. JH; here if \\S\\ < K then ^M onroe * s a P art ial Monroe-representation function). One can 
see that for a given set S, there may be many different (partial) iT-Monroe-representation functions 
that achieve optimal satisfaction; when we write ^Monroe' we mean > w.l.o.g., the particular one 
computed by the algorithm of Betzler et al. flU. 

3 Algorithms 

Let us now describe the algorithms that we will consider in this work. Some of our algorihtms can 
be applied both to Monroe's rule and to Chamberlin-Courtant's rule, while some are specific to only 
one of them. For each algorithm we will exactly specify for which rules it is applicable and, if it is 
applicable to both, what are the differences. 

While most of the algorithms described below are based on ones already given in the literature, 
in a number of cases we added heuristics on top of existing algorithms (which proved to be quite 
effective, as we will see later) and, in one case, provided a completely new theortical analysis. 
For each algorithm we will carefully describe what was already known in the literature, and which 
additions are due to this paper. 
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Throughout this section we assume we are given the following setting. A = {a±, . . . , a m } is a 
set of alternatives, a is an m-candidate PSF, N = [n] is a set of agents, each with a preference order 
over A, and K is a positive integer, K < rn (the size of the committee we want to elect). 

3.1 ILP Formulation (Monroe and CC) 

To measure the quality of our approximation algorithms, we compare their results against op- 
timal solutions that we obtain using integer linear programs (ILPs) that describe Monroe's and 
Chamberlin-Courant's rules. An ILP for Chamberlin-Courant's rule, for arbitraty PSF a, was pro- 
vided by Lu and Boutilier [5 ]; the analogous formulation for Monroe's rule was provided by Potthoff 
and Brams lfl3l . We used the GLPK 4.47 package (GNU Linear Programming Kit, version 4.47) to 
solve these ILPs, whenever it was possible to do so in reasonable time. 

3.2 Algorithms A, B, and C (Monroe) 

Skowron et al. fl5l have suggested and studied the following algorithm for Monroe's rule, which 
we will call Algorithm A. We start with an empty partial Monroe-representation function $ and we 
execute K iterations. In each iteration we do the following: 

1. For each alternative a € A that does not yet represent any agents, we compute the maximal 
satisfaction that some not-yet-represented \ agents derive from being represented by a (we call 
this number score(a) and we refer to these agents as bests(a)). 

2. We pick an alternative a with maximum score (a) and extend $ by assigning a to represent 
agents in bests (a). 

This algorithm clearly works in polynomial time. Skowron et al. lfl"5l have shown that for ag orda it 
finds a solution whose satisfaction is at least a (1 — 2 (m-i) ~~ f ract i° n of a (possibly nonexistent) 
perfect solution, where each agent is represented by his or her top preference (Hk is the K'th 
harmonic number, i.e., Hk = Yli=i J = ©(logiv")). This suggests that the algorithm performs 
best in elections where the size of the committee we seek is relatively small with respect to the 
number of alternatives. 

Based on Algorithm A we have derived Algorithm B. The only difference is that after complet- 
ing the operation of Algorithm A, we take the set S of alternatives that were assigned to represent 
some agents by Algorithm A, and replace function <I> with function ^Monroe' that optimally reas- 
signs the alternatives to the voters. This very simple heuristic turned out to noticeably improve the 
results of the algorithm in practice (and, of course, the approximation guarantees carry over from 
Algorithm A to Algorithm B). 

Algorithm C is a further heuristic improvement over Algorithm B. This time the idea is that 
instead of keeping only one partial function <3>, we keep a list of up to d partial representation func- 
tions, where d is a parameter of the algorithm. At each iteration, given these d partial representation 
functions, for each $ of them and for each alternative a that does not yet have agents assigned to by 
this we compute an optimal extension of this $ that assigns agents to a. As a result we obtain 
possibly more than d (partial) representation functions. For the next iteration we keep those d of 
them that have highest satisfaction. 
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Figured) The pseudocode for Algorithm C. 
Notation: $ <— a map defining a (partial) representation function, iteratively built by the algorithm. 
$^ <— the set of agents already represented by some alternative 
<£>^ the set of alternatives already used in the representation function. 
Par a list of partial representation functions 

Par = [} 
Par. push({}) 
for i <— 1 to K do 

newPar = [] 
for $ e Par do 

score <— {} 
6es£s «- {} 

foreach a, € A \ $^ do 

agents •(— sort TV \ «3> <- so that j ' -< k in agents 

=> posjfai) <pos k (ai) 
bests[ai] <— chose first elements of agents 
$' <- $ 

foreach j e bests [a;] do 

I $'[?'] «-Oj 

newPar .push($') 

sort newPar according to descending order of the total satisfaction of the assigned agents 
Par chose first d elements of newPar 
for $ e Par do 

$ <— compute the optimal representative function using an algorithm of Betzler et al. (2) for the 

set of winners $^ 
return the best representative function from Par 



We provide pseudocode for Algorithm C in Figure [T] If we take d = 1, we obtain Algorithm B. 
If we also disregard the last two lines prior to returning solution, we obtain Algorithm A. 

3.3 Algorithm GM (Monroe and CC) 

Algorithm GM (greedy marginal improvement) was introduced by Lu and Boutilier for the case 
of Chamberiin-Courant's rule. Here we generalize it to apply to Monroe's rule as well, and we 
show that it is a 1 — - approximation algorithm for a-Monroe. We point out that this is the first 
approximation result for Monroe rule that applies to all PSFs a (approximability results of Lu and 
Boutilier [5] did not apply to o-Monroe, and results of Skowron et al. lH5l applied to Monroe with 
Borda count PSF only). For the case of Monroe, the algorithm can also be viewed as an extension 
of Algorithm B. 

Let R be one of Monroe and CC. The algorithm proceeds as follows. We start with an emtpy 
set S. Then we execute K iterations. In each iteration we find an alternative a that is not assigned to 
agents yet, and maximizes the value <5^ u ^. (A certain disadvantage of this algorithm for the case 
of Monroe is that it requires a large number of computations of ^Monroe' wn i cn is a slow process 
based on min-cost/max-flow algorithm.) We provide the pseudocode for Algorithm GM in Figure |2] 

Theorem 1. Algorithm GM is an (1 — 1 / ' e)-approximation algorithm for the Monroe' e election 
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Figure|2j Pseudocode for Algorithm GM. 



Notation: R is either Monroe or CC. 
for i 1 to K do 

/iSu|o}> 

a <- argmax aGj4 \ s a($ fl J ) 
S ^SU{a} 
return 



problem for arbitrary positional scoring functions. 

Proof. The proof is based on the powerful result of Nemhauser et al. lITD . who have shown that 
greedy algorithms achieve 1 — - approximation ratio when used to optimize submodular functions. 
Let A be a set of alternatives, N = [n] a set of agents with preferences over A, a an 1 1 A || -candidate 
PSF, and K < \\A\\ the number of representatives that we want to elect. 

We consider function z : 2 — >• N defined, for each set S, S C A and ||5|| < K, as z(S) = 
a ( c ^ ) Monroc)- Clearly, z(S) is monotonic (that is, for each two sets A and B, if A C B and ||B|| < K 
then z(A) < z{B). The main part of the proof below is to show that z is submodular (we provide 
the definition below). 

Since argmax Sc ^ n S » =K z(S) is the set of winners of our election (under a-Monroe) and since 
Algorithm GM builds the solution iteratively by greedily extending initially empty set S so that 
each iteration increases the value of z(S) maximally, by the results of Nemhauser et al. ifTTTl we get 
that Algorithm GM is a (1 — -) -approximation algorithm. 

Let us now prove that z is submodular. That is, our goal is to show that for each two sets S and 
T, S C T, and each alternative a (£ T it holds that z{S U {a}) - z{S) > z(T U {a}) - z(T). First, 
we introduce a notion that generalizes the notion of a partial set of winners S. Let s : A — > N denote 
a function that assigns a capacity to each alternative (i.e., s gives a bound on the number of agents 
that a given alternative can represent). Intuitively, each set S, S C A, corresponds to the capacity 
function that assigns \^~\ to each alternative a G S and to each a ^ S. Given a capacity function 
s, we define a partial solution ^Monroe to ^ e one tnat maximizes the total satisfaction of the agents 
and that satisfies the capacity constraints: V ae 5||(<I>^ Ioni . oe )~ 1 (a)|| < s(a). To simplify notation, we 
write sL){a} to denote the function such that (sU{a}) (a) = s(a)+l and V a / g 5(sU{a})(a') = s(a'). 
(Analogously, we interpret s\{a} as subtracting one from the capacity for a; provided it is nonzero.) 
Also, by s < t we mean that V agj 4s(a) < t(a). We extend our function z to allow us to consider 
a subset of the agents only. For each subset N' of the agents and each capacity function s, we 
define z^i (s) to be the satisfaction of the agents in N' obtained under $ Monroe - ^ e wm now P rove 
a stronger variant of submodularity for our extended z. That is, we will show that for each two 
capacity functions s and t it holds that: 

s < t => zn(s U {a}) - z N (s) > z N (t U {a}) - z N (t) (1) 

Our proof is by induction on N. Clearly, Equation dU holds for N' = 0. Now, assuming that 
Equation £[]) holds for every N' C N we will prove its correctness for N. Let i denote an agent such 
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that ^Monroe W = a, (if there is no such agent then clearly the equation holds). Let a s = ^Monroe (*) 
and a t = $ Monroe (i). We have: 

z N (t U {a}) - z N (t) = a(poSj(a)) + z N \ {i] (t) - a(pos i (a t )) - z N \ {i} (t \ {a t }). 

We also have: 

z N (s U {a}) - z N (s) > a(poSj(a)) + 2Ar\{i}(s) - a(poSj(a s )) - z N \ {i} (s \ {a s }). 

Since ^Monroe describes an optimal representation function under the capacity restrictions t, we 
have that: 

a(pos i (a t )) + z N \ {i }(t \ a t ) > a(poSj(a s )) + z N \ {i} (t \ {a s }) 
Finally, from the inductive hypothesis for N' = N \ {i} we have: 

z N\{i}(s) - z N \ {i} (s \ {a s }) > z N \{ i} (t) - z N \ {i} (t \ {a s }) 
By combining these inequalities we get: 

z N (s U {a}) - z N (s) > a(poSj(a)) + z N \ {i} (s) - (a(pos i (a s )) + z N \ {i }{s \ {a s })) 

> a(poSj(a)) - a(poSi(a s )) + z N \ {i} (t) - z N \ {i} (t \ {a s }) 

> a(poSj(a)) + z N \ {i} (t) - a(pos i (a t )) - z N \ {i} (t \ {a t }) 
= z N (t U {a}) - z N (t) 

This completes the proof. □ 

3.4 Algorithm C (CC) 

This algorithm, introduced in this paper, proceeds like Algorithm GM for Chamberlin-Courant's 
rule, but in each iteration it keeps up to d (partial) CC -representation functions ^^C' ^ or distinct 
subsets S of alternatives (d is a parameter of the algorithm). In each iteration the algorithm extends 
each function <3?^ c by every possible alternative (obtaining 0(dm) new representation functions) 
and stores up to d of them, that obtain highest satisfaction. 

3.5 Algorithm P (CC) 

Algorithm P (position restriction) was introduced and studied by Skowron et al. 03]. The algorithm 
proceeds as follows. First, we consider a certain number x (specifically, x = [ m ^r^ ] , where W(x) 
is Lambert's W function, defined as the solution of equality x = W(x)e w ^)). Then, the algorithm 
tries to greedily find a cover of as many agents as possible with K alternatives (an alternative is 
said to cover a given agent if this agent ranks this alternative among top x positions). Skowron et 
al. lfT5l have shown that for ag orda this algorithm finds a solution that is at most 1 — 2W ^ RS > times 
worse than a perfect (possibly nonexistent) solution, where every agent is represented by his or her 
top-preferred alternative. The pseudocode for Algorithm P is presented in Figure [3] 
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Figure |3j Pseudocode for Algorithm P. 
Notation: We use the same notation as in AlgorithmQ] 

num_pos :E (a) <— \\{i 6 [n] \ : posi(a) < x}\\ 

(the number of not-yet assigned agents that rank alternative a in one of their first x positions) 

$ = {} 

for i <— 1 to K do 

a,i <— argmax a£yl ^^num_pos :r (a) 
foreach j e [n] \ do 
if posj(ai) < x then 

i m] <- at 

foreach j £ A \ $^ do 

a <— such server from $^ that \/ a > e q,^poSj(a) < poSj(a') 



3.6 Algorithm R (Monroe and CC) 

Algorithm R (random sampling) is based on picking the set of winners randomly and matching them 
optimally to the agents. Skowron et al. [ 15 ] have shown that if one chooses a set S of K alterna- 
tives uniformly at random, then for ag orda -Monroe, the expected satisfaction of «Borda( ( ^ ) Monroe) 

is i(l + — r- 1 rr— — r ) — e, and that one has to repeat this process ~ 512 |°g( 1 ~ A ) times, to 

reach probability A of achieving this satisfaction. For example, for A = 0.99 and e = 0.1 this algo- 
rithm would require to repeat the sampling process 340000/K times (each time executing a costly 
matching algorithm). This makes the algorithm impractical, especially for small instances (where 
K is low). Thus in our experimental evaluation we will consider the modification of the algorithm 
that repeats the sampling process only 100 times. 

Oren ITT21 has shown an analogous result for the case of Chamberlin-Courant's rule. 

3.7 Summary of the Algorithms 

We summarize the algorithms that we use in Table [T] In particular, the table clearly shows that for 
the case of Monroe, Algorithms B and C are not much slower than Algorithm A but offer a chance 
of improved peformance. Algorithm GM is intuitively even more appealing, but achieves this at 
the cost of high time complexity. For the case of Chamberlin-Courant's rule, it is unclear which 
of the algorithms to expect to be superior. One of the main goals of this paper is to establish if 
either of the presented algorithms clearly dominates the others. Our implementations are available 
at |http : //mimuw . edu . pi / ~ps2 1 9 737 /monroe /experiments . tar . gz| 

4 Experimental Data 

We have considered both real-life preference-aggregation data and synthetic data, generated accord- 
ing to a number of election models. 
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Algorithm 


Approximation ratio for Borda PSF 


Runtime 


Reference 


A 


i K-l H K 
1 2(m-l) K 


Kmn 


Skowron et al. 11151 


1 j 


qc in A 1 o A 

(Xa 111 rXLg. r\ 


T<Cm7i-\-0( <b^\ 


^L111l> UdUCl ) 


c 


as in Alg. A 


dKmn+dO(® s ) 


(this paper) 


GM 
R 


as in Alg. A for Borda PSF; 1 — ~ for others 


KmO{<$> s ) 


(this paper) 
Skowron et al. lfl5l 


P 


-, 2W(K) 


nmW(K) 


Skowron et al. lfl5l 


GM 




Kmn 


Lu and Boutilier 


C 
R 


as in Alg. GM 

(l-^L_)(l + i) 


dKm(n+\og dm) 

|log(l-A)| 
e 2 n 


(this paper) 
Oren flU 



Table 1 : A summary of the algorithms studied in this paper. The top of the table presents algorithms 
for Monroe's rule and the bottom for Chamberlin-Courant's rule. In column ' Approx." we give cur- 
rently known approximation ratio for the algorithm under Borda PSF, on profiles with m candidates 
and where the goal is to select a committee of size K. Here, 0{<& s ) = 0(n 2 (K + logn)) is the 
complexity of finding a partial representation function with the algorithm of Betzler et al. 0. 



4.1 Real-Life Data 

We have used real-life data regarding people's preference on sushi types, movies, college courses, 
and competitors' performance in figure-skating competitions. One of the major problems regarding 
real-life preference data is that either people express preferences over a very limited set of alterna- 
tives, or their preference orders are partial. To address the latter issue, for each such data set we 
complemented the partial orders to be total orders using the technique of Kamishima [4]. (The idea 
is to complete each preference order based on those reported preference orders that appear to be 
similar.) 

Some of our data sets contain a single profile, whereas the others contain multiple profiles. 
When preparing data for a given number m of candidates and a given number n of voters from a 
given data set, we used the following method: We first uniformly at random chose a profile within 
the data set, and then we randomly selected n voters and m candidates. We used preference orders 
of these n voters restricted to these m candidates. 

Sushi Preferneces. We used the set of preferences regarding sushi types collected by Kamishima 
filial Kamishima has collected two sets of preferences, which we call SI and S2. Data set SI 
contains complete rankings of 10 alternatives collected from 5000 voters. S2 contains partial 
rankings of 5000 voters over a set of 100 alternatives (each vote ranks 10 alternatives). We used 
Kamishima [4] technique to obtain total rankings. 

Movie Preferences. Following Mattei et al. (H, we have used the NetFlix data sell of movie 
preferences (we call it Mv). NetFlix data set contains ratings collected from about 480 thousand 

2 The sushi data set is available under the following url: |http : / / www . kamishima ■ net/ sushiTl 
3 http://www.netflixprize.com/ 
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distinct users regarding 18 thousand movies. The users rated movies by giving them a score between 
1 (bad) and 5 (good). The set contains about 100 million ratings. We have generated 50 profiles 
using the following method: For each profile we have randomly selected 300 movies, picked 10000 
users that ranked the highest number of the selected movies, and for each user we have extended his 
or her ratings to a complete preference order using the method of Kamishima 0. 

Course Preferences. Each year the students at the AGH University choose courses that they 
would like to attend. The students are offered a choice of six courses of which they have to at- 
tend three. Thus the students are asked to give an unordered set of their three top-preferred courses 
and a ranking of the remaining ones (in case too many students select a course, those with the 
highest GPA are enrolled and the remaining ones are moved to their less-preferred courses). In 
this data set, which we call Cr, we have 120 voters (students) and 6 alternatives (courses). How- 
ever, due to the nature of the data, instead of using Borda count PSF as the satisfaction measure, 
we have used the vector (3, 3, 3, 2, 1, 0). We made this data set publicly available under the url: 
http : //mimuw . edu . pi/ ~ps 21 97 37 /mon roe/ registration . tar . gz| 

Figure Skating. This data set, which we call Sk, contains preferences of the judges over the 
performances in a figure-skating competitions. The data set contains 48 profiles, each describing a 
single competition. Each profile contains preference orders of 9 judges over about 20 participants. 
The competitions include European skating championships, Olympics, World Junior, and World 
Championships, all from 19980- (Note that while in figure skating judges provide numerical scores, 
this data set is preprocessed to contain preference orders. ) 

4.2 Synthetic Data 

For our tests, we have also used profiles generated using three well-known distributions of prefer- 
ence orders. 

Impartial Culture. Under impartial culture model of preferences (which we denote IC), for a 
given set A of alternatives, each voter's preference order is drawn uniformly at random from the 
set of all possible total orders over A. While not very realistic, profiles generated using impartial 
culture model are a standard testbed of election-related algorithms. 

Polya-Eggenberger Urn Model. Following Walsh [16], we have used the Polya-Eggenberger urn 
model [ 1 ] (which we denote Ur). In this model we generate votes as follows. We have a set A of 
m alternatives and an urn that initially contains all ml preference orders over A. To generate a 
vote, we simply randomly pick one from the urn (this is our generated vote), and then — to simulate 
correlation between voters — we return a copies of this vote to the urn. When generating an election 
with m candidates using the urn model, we have set the parameter a so that A = 0.05 (Walsh lfi~6ll 
calls this parameter b; we mention that other authors use much higher values of b but we felt that 
too high a value of b leads to a much too strong a correlation between votes). 

Generalized Mallow's Model. We refer to this data set as Ml. Let >- and >-' be two preference 
orders over some alternative set A. Kendal-Tau distance between y and >-', denoted djc{>~, >-')> 

4 This data set is available under the following url: http : //rangevoting . org/SkateDatal 998 . txt| 



11 



is defined as the number of pairs of candidates x, y € A such that either x >- y A y x or 

y >~ x A x >-' y. 

Under Mallow's distribution of preferences ||7] we are given two parameters: A center prefer- 
ence order >- and a number <p between and 1. The model says that the probability of generating 
preference order >-' is proportional to the value (f> d K{>-,>- )_ To generate preference orders following 
Mallow's distribution, we use the algorithm given by Lu and Boutilier ||6)- 

In our experiments, we have used a mixture of Mallow's models. Let A be a set of al- 
ternatives and let a be a positive integer. This mixture model is parametrized by three vec- 
tors, A = (Ai,...,A a ) (where each Aj, 1 < i < a is between and 1, and Ylt=i ^1 = 1)> 
= (0i, . . . , cj) a ) (where each fa, 1 < i < a, is a number between and 1), and IT = (>-i, . . . , y a ) 
(where each 1 < i < a, is a preference order over A). To generate a vote, we pick a random in- 
teger i, 1 < i < a (each i is chosen with probability Aj), and then generate the vote using Mallow's 
model with parameters (>~i,fa). 

For our experiments we have used a = 5, and we have generated vectors A, <£, and II uniformly 
at random. 

5 Experiments 

In this section we present the results of the evaluation of algorithms from Section |3]on the data sets 
from Section H] In all cases, except for the college courses data set, we have used Borda PSF to 
measure voter satisfaction. For the case of the courses data set, we have used vector (3, 3, 3,2,1, 0). 

We have conducted three sets of experiments. First, we have tested all our algorithms on rela- 
tively small elections (up to 10 candidates, up to 100 agents). In this case we were able to compare 
our algorithms' solutions with the optimal ones. (To obtain the optimal solutions we were using the 
ILP formulations and GLPK's ILP solver.) Thus we report the quality of our algorithms as the av- 
erage of fractions C/C opt , where C is the satisfaction obtained by a respective algorithm and C opt 
is the satisfaction in the optimal solution. For each algorithm and data set, we also report the aver- 
age fraction C/C^ ea \, where Cid ea i is the satisfaction that the voters would have obtained if each 
of them were matched to his or her most preferred alternative. In our further experiments, where 
we consider larger elections, we were not able to compute optimal solutions, but fraction C/Cid ea i 
gives a lower bound for C/C opt . We report this value for small elections so that we see an example 
of relation between C/C opt and C/Cid ea i an d so that we can compare the results for small elections 
with the results for the larger ones. Further, for the case of Borda PSF the C/Cid ea i fraction has 
a very natural interpretation: If its value is a (for a given solution), then, on the average, in this 
solution each voter is matched to an alternative that he or she prefers to (m — l)a alternatives. 

In our second set of experiments we have run our algorithms on large elections (thousands 
of agents, hundreds of alternatives), coming either from the NetFlix data set or generated by us 
using one of our models. Here we reported the average fraction C/Cid ea i only. We have analyzed 
the quality of the solutions as a function of the number of agents, the number of candidates, and 
the relative number of winners (fraction K/m). (This last set of results is particularly interesting 
because in addition to measuring the quality of our algorithms, it allows one to asses the size of a 
committee one should seek if a given average agent satisfaction is to be obtained). 
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Monroe 


CC 




A 


B 


C 


GM 


R 


c 


GM 


P 


R 


SI 


0.94 


0.99 


PS 1.0 


0.99 


0.99 


1.0 


ps 1.0 


0.99 


0.99 


S2 


0.95 


0.99 


1.0 


ps 1.0 


0.99 


1.0 


ps 1.0 


0.98 


0.99 


Mv 


0.96 


pa 1.0 


1.0 


ps 1.0 


0.98 


1.0 


ps 1.0 


0.96 


ps 1.0 


Cr 


0.98 


0.99 


1.0 


ps 1.0 


0.99 


1.0 


ps 1.0 


1.0 


ps 1.0 


Sk 


0.99 


ps 1.0 


1.0 


ps 1.0 


0.94 


1.0 


ps 1.0 


0.85 


0.99 


IC 


0.94 


0.99 


ps 1.0 


0.99 


0.99 


1.0 


ps 1.0 


0.99 


0.99 


Ml 


0.94 


0.99 


1.0 


0.99 


0.99 


1.0 


ps 1.0 


0.99 


0.99 


Ur 


0.95 


0.99 


PS 1.0 


0.99 


0.99 


1.0 


0.99 


0.97 


0.99 



Table 2: The average quality of the algorithms compared with the optimal solution (C/C opt ) for the 
small instances of data and for K = 3. 
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ps 1.0 


0.98 


ps 1.0 


Mv 


0.95 


0.99 


1.0 


pa 1.0 


0.98 


1.0 


pa 1.0 


0.97 


pa 1.0 


Cr 


0.96 


ps 1.0 


1.0 


ps 1.0 


0.99 


1.0 


1.0 


1.0 


1.00 


Sk 


0.99 


ps 1.0 


1.0 


pa 1.0 


0.88 


1.0 


1.0 


0.91 


ps 1.0 


IC 


0.95 


0.99 


« 1.0 


0.99 


0.99 


1.0 


pa 1.0 


0.99 


0.99 


Ml 


0.95 


0.99 


pa 1.0 


0.99 


0.99 


1.0 


ps 1.0 


0.98 


0.99 


Ur 


0.96 


0.99 


pa 1.0 


0.99 


ps 1.0 


1.0 


ps 1.0 


0.96 


0.99 



Table 3: The average quality of the algorithms compared with the optimal solution (C/C op t) for the 
small instances of data and for K = 6. 



In the third set of experiments we have measured running times of our algorithms and of the 
ILP solver. 

5.1 Evaluation on Small Instances 

We now present the results of our experiments on small elections. For each data set, we generated 
elections with the number of agents n = 100 (n = 9 for data set Sk because there are only 9 
voters there) and with the number of alternatives m = 10 (m = 6 for data set Cr because there 
are only 6 alternatives there) using the method described in Section 14.11 for the real-life data sets, 
and in the natural obvious way for synthetic data. For each algorithm and for each data set we ran 
500 experiments on different instances for K = 3 (for the Cr data set we used K = 2) and 500 
experiments for K = 6 (for Cr we set K = 4). For Algorithms C we set the parameter d = 15. 
The results (average fractions C/C op t and C/Cideal) for K = 3 are given in Tables |2] and HI the 
results for K = 6 are given in Tables [3] and [5] (they are almost identical as for K = 3). For each 
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0.91 
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0.93 
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0.85 
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0.9 
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0.92 


0.90 


0.89 


0.94 


Ur 


0.8 


0.85 


0.86 


0.87 


0.85 


0.9 


0.87 


0.87 


0.89 



Table 4: The average quality of the algorithms compared with the simple lower bound (C/Cjdeai) 
for the small instances of data and for K = 3. 
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Cr 


0.95 
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« 1.0 


IC 
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0.95 


0.95 
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0.95 


0.96 


0.96 


0.95 


0.95 


Ml 


0.89 


0.94 


0.94 


0.94 


0.93 


0.97 


0.98 


0.95 


0.98 


Ur 


0.91 


0.95 


0.95 


0.94 


0.95 


0.98 


0.98 


0.94 


0.97 



Table 5: The average quality of the algorithms compared with the simple lower bound (C/Cideal) 
for the small instances of data and for K = 6. 



experiment in this section we also computed the standard deviation; it was always on the order of 

0. 01. The results lead to the following conclusions: 

1. Even Algorithm A obtains very good results, but nonetheless Algorithms B and C improve no- 
ticeably upon Algorithm A. In particular, Algorithm C (for d = 15) obtains the highest satisfaction 
on all data sets and in almost all cases was able to find an optimal solution. 

2. Algorithm R gives slightly worse solutions than Algorithm C. 

3. The quality of the algorithms does not depend on the data set used for verification (the only 
exception is Algorithm R for Monroe's system on data set Sk; however Sk has only 9 voters so it 
can be viewed as a border case). 
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5.2 Evaluation on Larger Instances 

For experiments on larger instances we needed data sets with at least n = 10000 agents. Thus we 
used the NetFlix data set and synthetic data. (Additionally, we run the subset of experiments (for 
n < 5000) also for the S2 data set.) For Monroe's rule we present results for Algorithm A, Algo- 
rithm C, and Algorithm R, and for Chamberlin-Courant's rule we present results for Algorithm C 
and Algorithm R. We limit the set of algorithms for the sake of the clarity of the presentation. For 
Monroe we chose Algorithm A because it is the simplest and the fastest one, Algorithm C because 
it is the best generalization of Algorithm A that we were able to run in reasonable time, and Algo- 
rithm R to compare a randomized algorithm to deterministic ones. For Chamberlin-Courant's rule 
we chose Algorithm C because it is, intuitively, the best one, and we chose Algorithm R for the 
same reason as in the case of Monroe. Further, we present results for the NetFlix data set and for 
the urn model only. We chose these data sets because the urn model results turned out to be the 
worst ones among the synthetic data sets, and the NetFlix data set is our only large real-life data set. 

First, for each data set and for each algorithm we fixed the value of m and K and for each 
n ranging from 1000 to 10000 with the step of 1000 we run 50 experiments. We repeated this 
procedure for 4 different combinations of m and K: (m = 10, K = 3), (m = 10, K = 6), 
(m = 100, K = 30) and (m = 100, K = 60). We measured the statistical correlation between 
the number of voters and the quality of the algorithms C/Cideal- The ANOVA test in most cases 
showed that there is no such correlation. The only exception was S2 data set, for which we obtained 
an almost negligible correlation. For example, for (m = 10, K = 3) Algorithm C under data set 
S2 for Monroe's system for n = 5000 gave C/Qdeai = 0.88, while for n = 100 (in the previous 
section) we got C/Cid ea i = 0.89. Thus we conclude that in practice the number of agents has 
almost no influence on the quality of the results provided by our algorithms. 

Next, we fixed the number of voters n = 1000 and the ratio K/m = 0.3, and for each m 
ranging from 30 to 300 with the step of 30 (naturally, as m changed, so did K to maintain the ratio 
K/m), we ran 50 experiments. We repeated this procedure for K/m = 0.6. The relation between 
m and C/Qdeai for Mv and Ur, under both Monroe's rule and Chamberlin-Courant's rule, is given 
in Figures @] and [5] (the results for K/m = 0.6 look similar). 

Finally, we fixed n = 1000 and m = 100, and for each K/m ranging from 0.1 and 0.5 with the 
step of 0.1 we run 50 experiments. The relation between the ratio K/m and the quality C/Cideal is 
presented in Figures [6] and [7] 

For the case of Chamberlin-Courant's system increasing the size of the committee we elect 
improves agent satisfaction: Since there are no constraints on the number of agents matched to a 
given alternative, larger committees mean more opportunities to satisfy the agents. For Monroe, 
larger committees may lead to lower total satisfaction. This happens if many agents like a particular 
alternative a lot, but only some of them can be matched to this alternative and others have to be 
matched to their less-preferred ones. Nonetheless, we see that Algorithm C achieves C / Cid ea i = 
0.925 even for K/m = 0.5 for the NetFlix data set. 

Our conclusions from these experiments are the following. For Monroe's rule, even Algorithm A 
achieves very good results. However, Algorithm C consistently achieves better ones (indeed, almost 
perfect ones). Randomized algorithms consistently do worse than our deterministic ones. 
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Figure 4: The relation between the number of alternatives m and the quality of the algorithms 
C/Cidcal f° r the Monroe's system. 
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Figure 5: The relation between the number of alternatives m and the quality of the algorithms 
C/Cidcai for the Chamberlin-Courant's system. 

5.3 Running time 

In our final set of experiments we have measured running times of our algorithms on the data set 
Mv. We have used a machine with Intel Pentium Dual T2310 1.46GHz processor and 1.5GB of 
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Figure 6: The relation between the ratio 2T/m and the quality of the algorithms C/Cideal for the 
Monroe's system. 
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Figure 7: The relation between the ratio K/m and the quality of the algorithms C/C^ e& \ for the 
Chamberlin-Courant's system. 



RAM. In Figure[8]we show the running time of GLPK ILP solver for Monroe's and for Chamberlin- 
Courant's rules. These running times are already large for small instances and they are increasing 
exponentially with the number of voters. For Monroe's rule, even for K = 9, m = 30, n = 100 
some of the experiments timed out after 1 hour, and for K = 9, m = 30, n = 200 none of the 
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Figure 8: The running time of the standard ILP solver for the Monroe's and for the Chamberlin- 
Courant's systems. For the Monroe's system, for K = 9, m = 30, and for n > 200 none of the 
single algorithm execution finished within 1 day. 

experiments finished within one day. Thus we conclude that the real application of the ILP algorithm 
is very limited. Example running times of the other algorithms for some combinations of n, m, and 
K are presented in Table [6] 

6 Conclusions 

We have provided experimental evaluation of a number of algorithms (both known ones and their 
extensions) for computing the winners under Monroe's rule and under Chamberlin-Courant's rale. 
While finding winners under these rales is NP-hard H2I5I141 . it turned out that in practice we can 
obtain very high quality solutions using simple algorithms. Indeed, both for Monroe's rule and for 
Chamberlin-Courant's rule we recommend using Algorithm C (or Algorithm A on very large Mon- 
roe elections). We believe that our results mean that (approximations of) Monroe's and Chamberlin- 
Courant's rules can be used in practice. 
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grant DEC-201 1/03/B/ST6/01393, and by EU's Human Capital Program "National PhD Programme 
in Mathematical Sciences" carried out at the University of Warsaw. 




18 







m - 


= 10, K 


= 3 


m = 


= 10, K 


= 6 




n = 


2000 
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0.032 
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0.06 
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m = 


100, K 


= 60 




n = 


2000 


6000 


10000 


2000 


6000 
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8 
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38 
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64 
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419 




GM 
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2172 


5313 


929 


5107 


13420 




R 


41 


329 


830 


88 


608 


1661 




C 


4.3 


11 


19 


7.5 


19 


31 


u 


GM 


0.06 


0.2 


0.4 


0.09 


0.3 


0.7 


u 


P 


0.03 


0.1 


0.26 


0.03 


0.1 


0.2 




R 


0.06 


0.24 


0.45 


0.1 


0.4 


0.8 



Table 6: Example running times of the algorithms [in seconds]. 
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